Papers with word similarity tasks
A Self-Supervised Integration Method of Pretrained Language Models and Word Definitions (2023.findings-acl)
Copied to clipboard
| Challenge: | DefBERT integrates word definition modeling with word semantics in dictionaries. |
| Approach: | They propose a method that integrates pretrained models with word semantics in dictionaries. |
| Outcome: | The proposed method is effective on word similarity and short text classification tasks. |
Learning Lexical Subspaces in a Distributional Vector Space (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing word embeddings that can cluster distributionally related words are weak, but they can be used to cluster words that might not be semantically similar. |
| Approach: | They propose a framework that injects lexical-semantic relations into distributional word embeddings by defining subspaces of the distributional vector space in which a lexically related relation should hold. |
| Outcome: | The proposed framework outperforms existing systems on relatedness and hypernymy tasks while being competitive on word similarity tasks. |
Self-supervised Post-processing Method to Enrich Pretrained Word Vectors (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods that use external resources to make word vectors specialize depend on the lexicon. |
| Approach: | They propose a self-supervised extension of extrofitting by its own word vector distribution. |
| Outcome: | The proposed method improves word similarity embeddings on similarity tasks without external resources. |
Quantifying Context Overlap for Training Word Embeddings (D18-1)
Copied to clipboard
| Challenge: | Experimental results show that word embeddings can be improved using word embeds . word embedings are a popular form of natural language processing . |
| Approach: | They propose to estimate second order co-occurrence relations based on context overlap . they use the augmented data to enhance word embeddings learning . |
| Outcome: | The proposed model improves word vectors for word similarity and downstream NLP tasks. |
GNEG: Graph-Based Negative Sampling for word2vec (P18-2)
Copied to clipboard
| Challenge: | Generally speaking, negative sampling is the best choice for distributed word representation learning. |
| Approach: | They hypothesize that taking into account global, corpus-level information and generating a different noise distribution for each target word better satisfies the requirements of negative examples for each training word. |
| Outcome: | The proposed approach boosts the word analogy task by about 5% and improves the performance on word similarity tasks by about 11% compared to the baseline. |
Urdu Word Embeddings (L18-1)
Copied to clipboard
| Challenge: | Recent advances in distributional semantics have led to the rise of neural network-based models that use unsupervised learning to represent words as dense, distributed vectors, called 'word embeddings' embedders hold key to improving natural language processing for low-resource languages, since they require significant time and manpower. |
| Approach: | They train a skip-gram model on 140 million Urdu words to create the first large-scale word embeddings for the Urdu language. |
| Outcome: | The proposed models capture high degree of syntactic and semantic similarity between words and are able to generalize well on the Urdu translation task. |
Word2Box: Capturing Set-Theoretic Semantics of Words using Box Embeddings (2022.acl-long)
Copied to clipboard
Shib Dasgupta, Michael Boratko, Siddhartha Mishra, Shriya Atmakuri, Dhruvesh Patel, Xiang Li, Andrew McCallum
| Challenge: | Word2Box provides a set-theoretic training objective for learning word representations . word representation is not natural, all senses and contexts, levels of abstraction, variants and modifications which the word may represent are forced to be captured by mat t is nunc. |
| Approach: | They propose a fuzzy-set interpretation of box embeddings and learn box representations of words using a set-theoretic training objective. |
| Outcome: | The proposed model improves word similarity tasks on less common words. |
Retrofitting Word Representations for Unsupervised Sense Aware Word Similarities (L18-1)
Copied to clipboard
| Challenge: | Standard word embeddings lack the ability to distinguish senses of a word by projecting them to exactly one vector. |
| Approach: | They propose to retrofit standard word embeddings to produce sense-aware embeddable vectors using external resources as sense inventories. |
| Outcome: | The proposed method improves word similarity and relatedness scores on multiple word embeddings and established word similarities, sometimes up to an impressive margin of +0.15 Spearman correlation score. |
Coming to Your Senses: on Controls and Evaluation Sets in Polysemy Research (D18-1)
Copied to clipboard
| Challenge: | a prior art claim that sense-specific vectors provide an advantage over normal vectors is unfounded in two ways. |
| Approach: | They claim that sense-specific vectors provide an advantage over normal vectors due to the polysemy that they presumably represent. |
| Outcome: | The proposed results show that ground-truth polysemy degrades performance in word similarity tasks and that random assignment of words to senses improves performance. |
Definition Frames: Using Definitions for Hybrid Concept Representations (2020.coling-main)
Copied to clipboard
| Challenge: | a new hybrid representation is proposed that encodes semantic information extracted from definitions. |
| Approach: | They propose a matrix distributed representation extracted from definitions where each dimension is semantically interpretable. |
| Outcome: | The proposed representations have competitive performance with other distributional semantic approaches on word similarity tasks. |
Robust Backed-off Estimation of Out-of-Vocabulary Embeddings (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to solving out-of-vocabulary (OOV) words use subwords to represent oov words with a bag of subword. |
| Approach: | They propose a method to estimate oov word embeddings by referring to pre-trained word embeds for known words with similar surfaces to target ov words. |
| Outcome: | The proposed method improves word similarity tasks and biomedical tasks even with weak baselines. |